Conventionally, there has been a non-contact line-of-sight interface as one kind of interface for operating a computer. This line-of-sight interface detects a line-of-sight of a user as data by using a camera and a light source and operates icons and such on a computer screen by using the detected line-of-sight data. This line-of-sight interface films an eyeball while irradiating a light from a light source such as an infrared light on the eyeball and consequently detects direction data calculated from a reflected light such as an infrared light of a filmed image at a cornea surface as estimated line-of-sight data.
A different error for each user is generated between estimated line-of-sight data calculated by this technology and a real line-of-sight data by an actual user. The source for causing an error includes various elements such as an individual difference in an eyeball shape, optical refraction at a corneal surface, an individual difference regarding the position of fovea centralis and so on.
Therefore, a compensation parameter for each user is previously calculated in the aim to compensate an error in estimated line-of-sight data in regard to real line-of-sight data and a process called calibration performed by this compensation parameter on the estimated line-of-sight thus calculated is conducted.
The calibration processing is conducted by using a compensation parameter calculated from the difference between the actual direction data from an eyeball to each marker and the estimated line-of-sight data detected by having users gaze predetermined plural markers in turn and consequently detecting the estimated line-of-sight data when each marker is gazed at.
It becomes possible to detect direction data closer to an actual user gaze as line-of-sight data by conducting the calibration processing.
However, it is necessary to let a user gaze about five to twenty markers when a compensation parameter is generated for detecting high precision line-of-sight data and consequently user's burden was heavy. Under such a circumstance, a technology to decrease the calibration process to a one point marker is disclosed. (Refer to a patent literature 1 and non-patent literatures 1 to 3, for example.)
These techniques find an optical axis of an eyeball, which is an axis connecting a center of curvature of a cornea and a pupil center of a pupil from eyeball images by filming a reflected light out of a light source at a cornea surface and a pupil. And a discrepancy (including individual differences) between an optical axis of an eyeball and a visual axis (equivalent to a line-of-sight) is found by a calibration that gazes one point and consequently a line-of-sight is correctly found with shifting of the optical axis measured by a quantity of discrepancy. Because the position of fovea centralis inside an eyeball cannot be filmed by a camera from outside according to these technologies, it is difficult to reduce the number of point to be gazed at the time of calibration from one point.
The inventors of the present invention already proposed line-of-sight measurement equipment that does not require a calibration processing because of a constraint condition in which visual axes intersect on a display screen after measuring the optical axes of both eyeballs. (Refer to the patent literature 2)
The line-of-sight measurement device the inventors of the present invention propose is a measurement device that acquires an eyeball image by the reflected light out of a light source concerning a user watching a display screen using a camera, calculates an optical axis which is an axis connecting a center of curvature of a cornea and a pupil center of a pupil from eyeball images, then calculates a discrepancy between an optical axis and a visual axis being an axis connecting a fovea centralis and a center of curvature of a cornea by using the calculated optical axis, finds the visual axis by shifting the optical axis based on the discrepancy between the optical axis and the visual axis and calculates the user's fixation point on the screen as a cross point of the display and the visual axis.
On the other hand, when situations wherein a line-of-sight measurement is conducted for users who gaze in a long distance at operation of vehicles such as cars, trains and ships or a user who gazes in a long distance from an observation platform, not an interface for operating computers, are supposed as shown in FIG. 1, it is thought that users gaze somewhere in a long distance in the forward direction for most of the time. The arrow 1 in FIG. 1 shows the line-of-sight of an automobile driver 2, the line-of-sight of a train-operator 3 and the line-of-sight of a ship captain 4.
It would be very idealistic, regarding this kind of users, if a calibration is conducted naturally during visual line movement of gazing in a long distance without gazing somewhere specifically designated.
According to the technologies disclosed by the patent literature 1 and non-patent literatures 1 to 3 as mentioned above, there is a constraint such as the subject must gaze at least one predetermined point. Even under such a constraint, a calibration only for once at the first time would enough if an individual can be specified, however it would not be suitable for the case wherein unspecified large number of people were supposed to be measurement objects.
Also, according to the technology disclosed in the patent literature 2, automatic calibration is possible for a case wherein a display screen is gazed at, however in the case of operations of vehicles such as cars, trains and ships, the technology is not adequate because the time duration for the operators to gaze the nearby displays and so on is very short and the operators gaze at somewhere in the distance most of the time. Also, there is such a problem that one of the technologies disclosed in patent literature 2 utilizing the middle point of an intersection of optical axes of both eyeballs and a display can perform an automatic calibration within an instant (one frame), however precision of this technology becomes lower than that of the technology accompanied by a calibration with a close observation.
Furthermore, a technology for detecting a gaze direction is known (Refer to parent literature 3) by displaying a target on a virtual image surface in the forward direction of an operator using a head-up display for the operator to gaze. However, the calibration is conducted by using a target to be easily gazed at in this case, not by an automatic calibration during a natural visual line movement of a user.
Also, with regard to car driving, all the people do not necessarily own a car of themselves and a family member or a friend could drive that car. There could be an operator change during train operation and different person could operate. With regard to a ship, a plurality of persons exists in the bridge simultaneously and also there is a shift. Also, a possibility that an unexpected person operates and navigates the ship due to sudden illness cannot be denied. For this kind of circumstance, a technology that enables an automatic calibration is desired for the purpose of using a line-of-sight measurement device at all times as a safety device even under an operation by a user other than a designated user.